Comparison study of intraoperative surface acquisition methods on registration accuracy for soft-tissue surgical navigation.

Purpose: To study the difference between rigid registration and nonrigid registration using two forms of digitization (contact and noncontact) in human in vivo liver surgery. Approach: A Conoprobe device attachment and sterilization process was developed to enable prospective noncontact intraoperative acquisition of organ surface data in the operating room (OR). The noncontact Conoprobe digitization method was compared against stylus-based acquisition in the context of image-to-physical registration for image-guided surgical navigation. Data from n=10 patients undergoing liver resection were analyzed under an Institutional Review Board-approved study at Memorial Sloan Kettering Cancer Center. Organ surface coverage of each surface acquisition method was compared. Registration accuracies resulting from the acquisition techniques were compared for (1) rigid registration method (RRM), (2) model-based nonrigid registration method (NRM) using surface data only, and (3) NRM with one subsurface feature (vena cava) from tracked intraoperative ultrasound (NRM-VC). Novel vessel centerline and tumor targets were segmented and compared to their registered preoperative counterparts for accuracy validation. Results: Surface data coverage collected by stylus and Conoprobe were 24.6%±6.4% and 19.6%±5.0%, respectively. The average difference between stylus data and Conoprobe data using NRM was -1.05 mm and using NRM-VC was -1.42 mm, indicating the registrations to Conoprobe data performed worse than to stylus data with both NRM approaches. However, using the stylus and Conoprobe acquisition methods led to significant improvement of NRM-VC over RRM by average differences of 4.48 and 3.66 mm, respectively. Conclusion: The first use of a sterile-field amenable Conoprobe surface acquisition strategy in the OR is reported for open liver surgery. Under clinical conditions, the nonrigid registration significantly outperformed standard-of-care rigid registration, and acquisition by contact-based stylus and noncontact-based Conoprobe produced similar registration results. The accuracy benefits of noncontact surface acquisition with a Conoprobe are likely obscured by inferior data coverage and intrinsic noise within acquisition systems.

Soft Tissue Monitoring of the Surgical Field: Detection and Tracking of Breast Surface Deformations.

OBJECTIVE: Deformable object tracking is common in the computer vision field, with applications typically focusing on nonrigid shape detection and usually not requiring specific three-dimensional point localization. In surgical guidance however, accurate navigation is intrinsically linked to precise correspondence of tissue structure. This work presents a contactless, automated fiducial acquisition method using stereo video of the operating field to provide reliable three-dimensional fiducial localization for an image guidance framework in breast conserving surgery. METHODS: On n = 8 breasts from healthy volunteers, the breast surface was measured throughout the full range of arm motion in a supine mock-surgical position. Using hand-drawn inked fiducials, adaptive thresholding, and KAZE feature matching, precise three-dimensional fiducial locations were detected and tracked through tool interference, partial and complete marker occlusions, significant displacements and nonrigid shape distortions. RESULTS: Compared to digitization with a conventional optically tracked stylus, fiducials were automatically localized with 1.6 ± 0.5 mm accuracy and the two measurement methods did not significantly differ. The algorithm provided an average false discovery rate <0.1% with all cases' rates below 0.2%. On average, 85.6 ± 5.9% of visible fiducials were automatically detected and tracked, and 99.1 ± 1.1% of frames provided only true positive fiducial measurements, which indicates the algorithm achieves a data stream that can be used for reliable on-line registration. CONCLUSIONS: Tracking is robust to occlusions, displacements, and most shape distortions. SIGNIFICANCE: This work-flow friendly data collection method provides highly accurate and precise three-dimensional surface data to drive an image guidance system for breast conserving surgery.

Incorporating heterogeneity and anisotropy for surgical applications in breast deformation modeling.

BACKGROUND: Simulating soft-tissue breast deformations is of interest for many applications including image fusion, longitudinal registration, and image-guided surgery. For the surgical use case, positional changes cause breast deformations that compromise the use of preoperative imaging to inform tumor excision. Even when acquiring imaging in the supine position, which better reflects surgical presentation, deformations still occur due to arm motion and orientation changes. A biomechanical modeling approach to simulate supine breast deformations for surgical applications must be both accurate and compatible with the clinical workflow. METHODS: A supine MR breast imaging dataset from n = 11 healthy volunteers was used to simulate surgical deformations by acquiring images in arm-down and arm-up positions. Three linear-elastic modeling approaches with varying levels of complexity were used to predict deformations caused by this arm motion: a homogeneous isotropic model, a heterogeneous isotropic model, and a heterogeneous anisotropic model using a transverse-isotropic constitutive model. FINDINGS: The average target registration errors for subsurface anatomical features were 5.4 ± 1.5 mm for the homogeneous isotropic model, 5.3 ± 1.5 mm for the heterogeneous isotropic model, and 4.7 ± 1.4 mm for the heterogeneous anisotropic model. A statistically significant improvement in target registration error was observed between the heterogeneous anisotropic model and both the homogeneous and the heterogeneous isotropic models (P < 0.01). INTERPRETATION: While a model that fully incorporates all constitutive complexities of anatomical structure likely achieves the best accuracy, a computationally tractable heterogeneous anisotropic model provided significant improvement and may be applicable for image-guided breast surgeries.

Supine magnetic resonance image registration for breast surgery: insights on material mechanics.

Purpose: Breast conserving surgery (BCS) is a common procedure for early-stage breast cancer patients. Supine preoperative magnetic resonance (MR) breast imaging for visualizing tumor location and extent, while not standard for procedural guidance, is being explored since it more closely represents the surgical presentation compared to conventional diagnostic imaging positions. Despite this preoperative imaging position, deformation is still present between the supine imaging and surgical state. As a result, a fast and accurate image-to-physical registration approach is needed to realize image-guided breast surgery. Approach: In this study, three registration methods were investigated on healthy volunteers' breasts ( n = 11 ) with the supine arm-down position simulating preoperative imaging and supine arm-up position simulating intraoperative presentation. The registration methods included (1) point-based rigid registration using synthetic fiducials, (2) nonrigid biomechanical model-based registration using sparse data, and (3) a data-dense three-dimensional diffeomorphic image-based registration from the Advanced Normalization Tools (ANTs) repository. Additionally, deformation metrics (volume change and anisotropy) were calculated from the ANTs deformation field to better understand breast material mechanics. Results: The average target registration errors (TRE) were 10.4 ± 2.3 , 6.4 ± 1.5 , and 2.8 ± 1.3 mm (mean ± standard deviation) and the average fiducial registration errors (FRE) were 7.8 ± 1.7 , 2.5 ± 1.1 , and 3.1 ± 1.1 mm for the point-based rigid, nonrigid biomechanical, and ANTs registrations, respectively. The mechanics-based deformation metrics revealed an overall anisotropic tissue behavior and a statistically significant difference in volume change between glandular and adipose tissue, suggesting that nonrigid modeling methods may be improved by incorporating material heterogeneity and anisotropy. Conclusions: Overall, registration accuracy significantly improved with increasingly flexible and data-dense registration methods. Analysis of these outcomes may inform the future development of image guidance systems for lumpectomy procedures.

Tumor deformation correction for an image guidance system in breast conserving surgery.

Breast cancer is the most common cancer in women, and surgical resection is standard of care for the majority of breast cancer patients. Unfortunately, current reoperation rates are 10-29%. Uncertainty in lesion localization is one of the main factors contributing to these high reoperation rates. This work uses the linearized iterative boundary reconstruction approach to model patient breast deformation due to abduction of the ipsilateral arm. A preoperative supine magnetic resonance (MR) image was obtained with the patient's arms down near the torso. A mock intraoperative breast shape was measured from a supine MR image obtained with the patient's arm up near the head. Sparse data was subsampled from the full volumetric image to represent realistic intraoperative data collection: surface fiducial points, the intra-fiducial skin surface, and the chest wall as measured with 7 tracked ultrasound images. The deformed preoperative arm-down data was compared to the ground truth arm-up data. From rigid registration to model correction the tumor centroid distance improves from 7.3 mm to 3.3 mm, average surface fiducial error across 9 synthetic fiducials and the nipple improves from 7.4 ± 2.2 to 1.3 ± 0.7, and average subsurface error across 14 corresponding features improves from 6.2 ± 1.4 mm to 3.5 ± 1.1 mm. Using preoperative supine MR imaging and sparse data in the deformed position, this modeling framework can correct for breast shape changes between imaging and surgery to more accurately predict intraoperative position of the tumor as well as 10 surface fiducials and 14 subsurface features.

Breast image registration for surgery: Insights on material mechanics modeling.

Breast conserving surgery (BCS) is a common procedure for early-stage breast cancer patients. Supine preoperative magnetic resonance (MR) breast imaging for visualizing tumor location and extent, while not standard for procedural guidance, more closely represents the surgical presentation compared to conventional diagnostic pendant positioning. Optimal utilization for surgical guidance, however, requires a fast and accurate image-to-physical registration from preoperative imaging to intraoperative surgical presentation. In this study, three registration methods were investigated on healthy volunteers' breasts (n=11) with the arm-down position simulating preoperative imaging and arm-up position simulating intraoperative data. The registration methods included: (1) point-based rigid registration using synthetic fiducials, (2) non-rigid biomechanical model-based registration using sparse data, and (3) a data-dense 3D diffeomorphic image-based registration from the Advanced Normalization Tools (ANTs) repository. The average target registration errors (TRE) were 10.4 ± 2.3, 6.4 ± 1.5, and 2.8 ± 1.3 mm (mean ± standard deviation) and the average fiducial registration errors (FRE) were 7.8 ± 1.7, 2.5 ± 1.1, and 3.1 ± 1.1 mm (mean ± standard deviation) for the point-based rigid, nonrigid biomechanical, and ANTs registrations, respectively. Additionally, common mechanics-based deformation metrics (volume change and anisotropy) were calculated from the ANTs deformation field. The average metrics revealed anisotropic tissue behavior and a statistical difference in volume change between glandular and adipose tissue, suggesting that nonrigid modeling methods may be improved by incorporating material heterogeneity and anisotropy. Overall, registration accuracy significantly improved with increasingly flexible registration methods, which may inform future development of image guidance systems for lumpectomy procedures.

Computational Imaging to Compensate for Soft-Tissue Deformations in Image-Guided Breast Conserving Surgery.

OBJECTIVE: During breast conserving surgery (BCS), magnetic resonance (MR) images aligned to accurately display intraoperative lesion locations can offer improved understanding of tumor extent and position relative to breast anatomy. Unfortunately, even under consistent supine conditions, soft tissue deformation compromises image-to-physical alignment and results in positional errors. METHODS: A finite element inverse modeling technique has been developed to nonrigidly register preoperative supine MR imaging data to the surgical scene for improved localization accuracy during surgery. Registration is driven using sparse data compatible with acquisition during BCS, including corresponding surface fiducials, sparse chest wall contours, and the intra-fiducial skin surface. Deformation predictions were evaluated at surface fiducial locations and subsurface tissue features that were expertly identified and tracked. Among n = 7 different human subjects, an average of 22 ± 3 distributed subsurface targets were analyzed in each breast volume. RESULTS: The average target registration error (TRE) decreased significantly when comparing rigid registration to this nonrigid approach (10.4 ± 2.3 mm vs 6.3 ± 1.4 mm TRE, respectively). When including a single subsurface feature as additional input data, the TRE significantly improved further (4.2 ± 1.0 mm TRE), and in a region of interest within 15 mm of a mock biopsy clip TRE was 3.9 ± 0.9 mm. CONCLUSION: These results demonstrate accurate breast deformation estimates based on sparse-data-driven model predictions. SIGNIFICANCE: The data suggest that a computational imaging approach can account for image-to-surgery shape changes to enhance surgical guidance during BCS.

Impact of deformation on a supine-positioned image-guided breast surgery approach.

PURPOSE: To reduce reoperation rates for image-guided breast-conserving surgery, the enhanced sensitivity of magnetic resonance (MR) supine imaging may be leveraged. However, accurate tissue correspondence between images and their physical counterpart in the surgical presentation is challenging due to breast deformations (e.g., from patient/arm position changes, and operating room table rotation differences). In this study, standard rigid registration methods are employed and tissue deformation is characterized. METHODS: On n = 10 healthy breasts, surface displacements were measured by comparing intraoperative fiducial locations as the arm was moved from conventional MR scanning positions (arm-down and arm-up) to the laterally extended surgical configuration. Supine MR images in the arm-down and arm-up positions were registered to mock intraoperative presentations. RESULTS: Breast displacements from a supine MR imaging configuration to a mock surgical presentation were 28.9 ± 9.2 mm with shifts occurring primarily in the inferior/superior direction. With respect to supine MR to surgical alignment, the average fiducial, target, and maximum target registration errors were 9.0 ± 1.7 mm, 9.3 ± 1.7 mm, and 20.0 ± 7.6 mm, respectively. Even when maintaining similar arm positions in the MR image and mock surgery, the respective averages were 6.0 ± 1.0 mm, 6.5 ± 1.1 mm, and 12.5 ± 2.8 mm. CONCLUSION: From supine MR positioning to surgical presentation, the breast undergoes large displacements (9.9-70.1 mm). The data also suggest that significant nonrigid deformations (9.3 ± 1.7 mm with 20.0 mm average maximum) exist that need to be considered in image guidance and modeling applications.

Textual fiducial detection in breast conserving surgery for a near-real time image guidance system.

Breast cancer is the most common cancer in American women, and is the second most deadly. Current guidance approaches for breast cancer surgery provide distance to a seed implanted near the tumor centroid. Large deformations between preoperative imaging and surgical presentation, coupled with the lack of tumor extent information leads to difficulty in ensuring complete tumor resection. Here we propose a novel image guidance platform that utilizes character-based fiducials for easy detection and small fiducial points for precise and accurate localization. Our system is work-flow friendly, and near-real time with use of stereo cameras for surface acquisition. Using simple image processing techniques, the proposed technique can localize fiducials and character labels, providing updates without relying on video history. Character based fiducial labels can be recognized and used to determine correspondence between left and right images in a pair of stereo cameras, and frame to frame in a sequence of images during a procedure. Letters can be recognized with 89% accuracy using the MATLAB built in optical character recognition function, and an average of 81% of points can be accurately labeled and localized. The stereo camera system can determine surface points with accuracy below 2mm when compared to optically tracked stylus points. These surface points are incorporated to a four-panel guidance display that includes preoperative supine MR, tracked ultrasound, and a model view of the breast and tumor with respect to optically tracked instrumentation.

A system for automatic monitoring of surgical instruments and dynamic, non-rigid surface deformations in breast cancer surgery.

When negative tumor margins are achieved at the time of resection, breast conserving therapy (lumpectomy followed with radiation therapy) offers patients improved cosmetic outcomes and quality of life with equivalent survival outcomes to mastectomy. However, high reoperation rates ranging 10-59% continue to challenge adoption and suggest that improved intraoperative tumor localization is a pressing need. We propose to couple an optical tracker and stereo camera system for automated monitoring of surgical instruments and non-rigid breast surface deformations. A bracket was designed to rigidly pair an optical tracker with a stereo camera, optimizing overlap volume. Utilizing both devices allowed for precise instrument tracking of multiple objects with reliable, workflow friendly tracking of dynamic breast movements. Computer vision techniques were employed to automatically track fiducials, requiring one-time initialization with bounding boxes in stereo camera images. Point based rigid registration was performed between fiducial locations triangulated from stereo camera images and fiducial locations recorded with an optically tracked stylus. We measured fiducial registration error (FRE) and target registration error (TRE) with two different stereo camera devices using a phantom breast with five fiducials. Average FREs of 2.7 ± 0.4 mm and 2.4 ± 0.6 mm with each stereo-camera device demonstrate considerable promise for this approach in monitoring the surgical field. Automated tracking was shown to reduce error when compared to manually selected fiducial locations in stereo camera image-based localization. The proposed instrumentation framework demonstrated potential for the continuous measurement of surgical instruments in relation to the dynamic deformations of a breast during lumpectomy.

Directed self-assembly software for single cell deposition.

Laser direct-write (LDW) bioprinting methods offer a diverse set of tools to design experiments, fabricate tissue constructs and to cellular microenvironments all in a CAD/CAM manner. To date, we have just scratched the surface of the system's potential and for LDW to be utilized to its fullest, there are many distinct hardware and software components that must be integrated and communicate seamlessly. In this perspective article, we detail the development of novel graphical user interface (GUI) software to improve LDW capability and functionality. The main modules in the control software correspond to cell transfer, microbead fabrication, and micromachining. The modules make the control of each of these features, and the management of printing programs that utilize one or more features, to be facile. The software also addresses problems related to construct scale-up, print speed, experimental conditions, and management of sensor data. The control software and possibilities for integrated sensor data are presented.